Pre-alpha MVP 0.0 (systemd-nspawn container)
This issue is for tracking the necessary changes and targets that need to be met before we can present a MVP Pre-Alpha that people can use in a non-bootable, self-hosting systemd-nspawn container.
Note that this self-hosting systemd-nspawn container will also need to act as our own build infra, which means that it needs to be in a decent state. By doing things this way, we defer the extra maintenance work associated with hardware support to later and can run our infra on whatever distro is deemed suitable (and so can users/potential investors).
Huge Refactor
-
Un-jobbify moss (serpent-os/core/moss#1 (closed)) -
Improve Storage in moss-format -
Add xxhash bindings with a specific focus on getting XXH3_64BIT and XXH3_128BIT working (serpent-os/dlang/xxhash-d#1 (closed)) -
Hash function improvements -
Switch content store hash to XXH3_128bit for simultaneous space savings and speed gains (serpent-os/core/moss-deps#1 (closed)) -
Switch CRC64ISO to XXH3_64bit for speed gains (serpent-os/core/moss-format#2 (closed))
-
-
-
moss-deps refactoring -
Refactor for elf.d ELFNote functionality (serpent-os/core/moss-deps@ab508fce + serpent-os/dlang/elf-d@e7dc6696)
-
-
Existing design/implementation cleanup pass (huge refactor) -
Replace InstallDB with MetadataDB -
Create ActivePackagesPlugin to filter MetadatadB by "active" state -
Add moss-format encoding/decoding of Dependency() and Provider() -
Add basic flags to RegistryItem (Installed/Available) -
Rename moss info
tomoss inspect
(serpent-os/core/moss@50fb53a2)
-
Dependency Management
-
Solidify Deps -
Add dependency solver via directed acyclic graph -
Add support for breaking U <-> V cycles in digraph -
Allow experimenting with local .stone files for fleshing out deps (serpent-os/core/moss#2 (closed)) -
Add interpreter type deps -
Emit Interpreter providers for libc packages ( ld-*.so
) -
Emit Interpreter compat providers for ld-*.so
symlinks
-
-
Add shared library deps
-
-
Solidify Transactions -
Add support for adding/removing packages locally with moss -
Add support for removing revdeps whose deps are no longer satisfied (via transposed subgraph resolution) -
Add support for moss list installed
-
Add support for moss info
-
Ensure that ProviderType.name is unique in a Transaction -
Uncomplicate Transaction to allow set manipulation and final result topologically sorted
-
Repository Enabling
-
Create web-facing repository and build management tools (scale-out infra) -
Summit (dashboard + build controller + group/user management) -
Avalanche (distributed builder management) -
Vessel (package fetching and indexing tool) -
Create auth routines -
Add ability to enrol Avalanche builders into Summit build controller -
Add ability to manage namespaces (1:1 relationship to GL groups/recipe repos)
-
-
Create first pass moss index repo -
Create LMDB D bindings and make moss-db our D API (serpent-os/dlang/lmdb-d#1 (closed)) -
Set up a RepoDB using a RepoReader and plugged into the RepoReader -
Add repository index support via MetaDB subclass + plugin API -
Add shared moss fetch module that uses curl to fetch files in parallel -
Ensure bootstrap-scripts build to enable parallel work by the team (We have a set of packages which are shared) -
Refresh manifest output for new changes -
Add a desired-system-configuration-set notion that the resolver can fulfill -
Repo configuration: Create a way to configure and layer repos (including local repo)
-
-
Create bootstrap packages -
Create a clever baselayout package that relies on/ties into moss/boulder denylist hard packaging errors -
Add support for --unconfined
builds inboulder
,moss
and.stone
payloads. Useful for creating and installing bootstrap packages. Print scary warnings when unconfined .stones are encountered and when the flag is specified. -
Add support for git
upstreams (see https://gitlab.com/serpent-os/dlang/libgit2-d)
-
Polish and Productisation
Performance related
-
Format optimisations (serpent-os/core/moss-format#3 (closed)) -
Review Directory entries in LayoutPayload either: -
Store only special dirs and recreate list from file/symlink/node path
-
-
Always store empty dirs or ones with non-standard permissions (standard is 00755 which is 4755 internally in DLang)
-
-
zstd
bindings: Create our own zstd Dlang bindings which support--long
mode (serpent-os/dlang/zstd-d#1) -
moss-format
: Reuse compression handlers to save large unnecessary allocations in upcoming zstd longmode usage -
moss-deps
: Split up for less expensive linking (serpent-os/core/moss-deps#2) -
moss-db
: Check that our performance expectations re. using LMDB D bindings hold -
moss
improvements:-
Improve ArchiveCacher performance ( copy_file_range
, etc) -
Improve RootConstructor performance ( buildPath
,hardLink
, etc)-
Remove use of buildPath
switching out withjoin
-
Cache subpaths that are used a lot ( auto rootfsDir
already stores the state path) (serpent-os/core/moss@f8866680) -
Consider reducing duplicate calculations for Layout
andAttrs
(rootconstructor.d
) -
Parallel option? (but making dirs first)
-
-
Minimise unnecessary I/O calls to/with the store ( chmod
etc)
-
-
Fix strace
file loading that takes way too long:-
strace clang --version
<- tries to load a bunch of gcc dirs... -
strace ls
<- locale, ld.so.cache and gconv loading
-
Packaging related
-
Tooling - Fix up packaging issues -
Ability to create debug packages (with debug key activated or presence of debug tuning enabled) -
Add definition so .so
Regular files aren't included in-devel
pkgs (likely depends on serpent-os/core/moss-format#5) -
Add shell-glob ( *.foo
) matching for adisablestrip
key which can be a single glob or list of globs -
Strip ELF files
-
-
baselayout
: Begin looking into how we want that to look-
/usr/lib64 -> lib
-
-
bash
: Add bootstrap-scripts bash prompt to bash vendor config -
bash
: Ensure a sane startup script loading order -
nano
: Add suitable ootb configuration w/syntax highlighting, convenience features and default indentation stuff (possibly nano 6.0 provides this) -
clang
:-
Target: minimize strace --trace=openat clang -v
output, it's disgraceful -
Add -DCMAKE_SKIP_RPATH:BOOL=ON \
to build -
Need ld.so.cache
for that to help. root owned / -
Add in the slightly reworked patch -
Add -DLLVM_PARALLEL_LINK_JOBS=1 \
to reduce memory consumption -
Do Valgrind analysis of clang
startup!
-
-
glibc
finalisation:-
Remove unused search paths (have a patch prepared) -
locale data - reduce and create "/var/cache/locale/locale-archive", "/usr/share/locale/locale.alias" (does it need to move to /var?) -
/usr/lib/locale/en_US.UTF-8
(doesn't exist) vs/usr/lib/locale/en_US.utf8
. Searches wrong path first
-
-
Creation of "/var/cache/ldconfig/ld.so.cache" -
iconvconfig - integrate into build
-
-
Update and review packages in the current system. Add in needed patches etc. -
Add ldc2
+ friends into core OS set for self-hosting capability and package dev+build inside of systemd-nspawn container
-
-
boulder
: Go through the various macros and check for potential improvements/additions -
moss-format
: Flesh out upgrade routine -
moss-ansi
: Try to get it working for TUI schmexiness -
benchmarking-tools
: Integrate into stone -
Add moss
blit benchmarkmoss rm
-
Add boulder
benchmark
Cleanup / Correctness related
-
Add a default set of code formatting options and create + add default .editorconfig
to promote/require formatting consistency (and avoid a source of bikeshedding). (See https://gitlab.com/serpent-os/core/serpent-style) -
Add serpent-style to all our projects/modules and run update-format.sh
(dfmt
-based) -
Add logging to all user-facing tools (moss, moss-container, boulder/mason) -
Create custom colour-capable logging (based on std.experimental.logger) -- see moss.core.logging -
Ensure that all format strings are checked at compile time
-
-
Ensure API documentation is good and current -
Ensure all functions documented -
Ensure that all modules use the new SumType-based error reporting method Ikey came up with -
Ensure that all user-facing tools are self-documenting with <tool> -h/--help
-
Eliminate all compiler warnings/errors -
Get rid of weird naming inconsistencies between projects (yo, it aint C# and it aint no Java) -
Rebase on latest dependent D modules -
Rebase on latest ldc2
Release
-
Build a systemd-nspawn image -
Ensure os-release
is generated from systemd-nspawn Ansible automation thingy
-
-
Seed image to community members for early feedback prior to GA -
Iterate a few times on pre-GA feedback -
Write blog post and tweet announcements as GA goes live